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The Supermarket Game 



Jiaming Xu and Bruce Hajek 



Abstract 

■ A supermarket game is considered with N FCFS queues with unit exponential service rate and global 

■ Poisson arrival rate A^A. Upon arrival each customer chooses a number of queues to be sampled uniformly at 
^ , random and joins the least loaded sampled queue. Customers are assumed to have cost for both waiting and 
^ , sampling, and they want to minimize their own expected total cost. We study the supermarket game in a mean 



field model that corresponds to the limit as A^ converges to infinity in the sense that (i) for a fixed symmetric 
customer strategy, the joint equilibrium distribution of any fixed number of queues converges as A^ — s> cxd to a 
product distribution determined by the mean field model and (ii) a Nash equilibrium for the mean field model 



C/2 , is an e-Nash equilibrium for the finite A^ model with A^ sufficiently large. It is shown that there always exists a 

O ■ 

Nash equilibrium for A < 1 and the Nash equilibrium is unique with homogeneous waiting cost for A^ < 1/2. 
Furthermore, we find that the action of sampling more queues by some customers has a positive externality 
_ on the other customers. 

oo" 
O 



(N 

o: 

CN ■ A. Problem Formulation 



I. Introduction 



> ■ Consider a supermarket game with FCFS queues with exponential service rate one, and global 
^ \ Poisson arrival rate N\. Assume A < 1 and let £ = {1, ... , Lmax}- Upon arrival, each customer can 
- - ■ choose a number L G £ of queues to be sampled uniformly at random, and the customer joins the 
sampled queue with the least number of customers, ties being resolved uniformly at random. Customers 
are assumed to have cost c per unit waiting time and Cg for sampling one queue. These cost parameters 
are the same for all the customers; heterogeneous waiting costs are considered in Section III-FI 

For a fixed customer i, if she chooses Li queues to sample, and all the other customers choose 
then by PASTA (Poisson arrival sees time average), the expected total cost of customer i is given by 

C(Li, L_i) = cE[iy(Li, L_,)] + c,L„ (1) 
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where K[W{Li, L^i)] is the expected waiting time (service time included) under the stationary distri- 
bution. The goal of customer i is to minimize her own expected total cost by choosing the optimal 
Lj. Note that her own expected waiting time also depends on the other customers' decisions, L_j. 
Therefore, it is a game among the customers and we call it the supermarket game, because often 
in supermarkets customers try to find counters with short queues to check out. In this paper, we 
characterize the Nash equilibria of the supermarket game and investigate the equilibrium behavior of 
customers. 

Since the supermarket game is symmetric in the customers, we limit ourselves to symmetric 
strategies. We call L* e C a pure strategy Nash equilibrium, if 



Since a pure strategy Nash equilibrium does not always exist, we are also interested in mixed strategy 
Nash equilibria. 

Let V{C) denote the set of all probability distributions over C. The mixed strategy Hi for customer 
i is simply a probability distribution in V{C), i.e., /ij(Lj) is the probability that customer i samples Lj 
queues. If all the other customers use the mixed strategy then the expected total cost of customer 
i using jii is given by 



where C{Li,ji_i) is the expected total cost of customer i choosing Lj given all the others choose the 

mixed strategy 

Define the best response correspondence for customer i as BR(yU_j) := argmin^j^ C{ni,H-i). The 
correspondence BR is a set-valued function from to subsets of V{C). We call /i* e 'Pi^) a 

mixed strategy Nash equilibrium if 



It is of interest to show the existence and uniqueness of a mixed strategy Nash equilibrium. However, 
the supermarket model with finite queues is difficult to analyze due to the correlation among queues. 
For instance, we have no closed form expression for the stationary distribution. Therefore, we study 
the supermarket game in a mean field model. We also justify the mean field model and show that for 
any e > 0, a Nash equilibrium in the mean field model is an e-Nash equilibrium in the finite N model 
with sufficiently large N. 



C{L\ V) < C{Li, L*), for all ^ e C. 




(2) 



C{ii*,ii*) < C{iii,ii*), for all in e r{C). 
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B. Motivation 

The supermarket game is relevant in the networking scenario where (1) the customers have options 
to choose which queue to join for their own interest; (2) there is no central dispatcher or tracker to route 
customers to particular servers; (3) global queue length information is not available and customers 
randomly choose a finite number of queues to probe; (4) there is cost associated with probing a server 
and waiting in a queue. For example, in a communication or transportation network, the flows or traffic 
can decide how many routes to be probed in order to find the route with small delay. Also, in large 
scale server farms or cloud clusters, the customers can decide how many servers to be probed to find 
the server with minimum delay. We address the following natural questions for these systems: What 
is the equilibrium behavior of the customers or flows? Is sampling or probing more queues by some 
customers beneficial or detrimental to the others? 

C. Main Results 

We study the supermarket game in a mean field model with homogeneous waiting cost and sampling 
cost. It is shown that: 

• There exists a mixed strategy Nash equilibrium for all arrival rates A < 1. 

• Nash equilibrium is unique for arrival rates < |. 

• Nash equilibrium is unique for all values of c and Cg if and only if a local monotonicity condition 
is satisfied. This condition is used to explore the uniqueness numerically for A^ > |. 

• Multiple Nash equilibria exist for a particular example with A = 0.999 and L^ax = 25. 

• Nash equilibrium is unique for A = 0.999 if the customers can only sample either one queue or 
two queues. 

• The action of sampling more queues by some customers has a positive externality on the other 
customers, which further implies that customers sample no more queues for any Nash equilibrium 
than for the socially optimal strategy. 

Then, we consider the heterogenous waiting cost case and prove the existence of pure strategy Nash 
equilibrium in that case. 

We also study the supermarket game with finite N . It is proved that: 

• A propagation of chaos result and a coupling result similar to the ones in f[\ and |l2l hold, and 
thus the joint equilibrium distribution of any fixed number of queues converges as — )■ oo to a 
product distribution determined by the mean field model. 

• A Nash equilibrium in the mean field model is an e-Nash equilibrium in the finite model. 
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• The action of sampling more queues by some customers has a nonnegative externality on the 
customers who only sample one queue. However, for customers sampling more than one queue, 
it is unknown whether the externality is positive. 

D. Related Work 

The supermarket game is based on the classical supermarket model in which customers sample L 
queues. The supermarket model is an abstraction of path selection in networks and task assignment at 
server farms, and has been extensively studied in the literature. Vvedenskaya et. al. |[3l shows that in 
the mean field model, the equilibrium queue sizes decay doubly exponentially for L > 2. Turner [|2l 
proves an interesting coupling result for fixed A^, which implies that the load in the network is handled 
better and more evenly as L increases. Graham proves a chaoticity result on the path space, and 
Mitzenmacher flU studies the model using Kurtz's theorem. Furthermore, Luczak and McDiarmid [[51 
prove that for L > 2, the maximum queue length scales as In Inn/ In L + 0(1). The recent paper by 
Bramson et. al. [i6| analyzes the supermarket model with general service time distributions. Ganesh 
et. al. [[71 studies a variant of the supermarket model, where the customers initially join an arbitrary 
server, but may switch to other servers later independently at random. 

In addition to the supermarket model, the mean field approach exploited in this paper, has also been 
used to analyze scheduling in queueing networks. For example, the CSMA algorithm in a wireless 
local area network is studied via mean field approach in [0. Also, a recent work [[9l investigates the 
performance tradeoff between centralized and distributed scheduling in a multi-server system for the 
mean field model. It shows that even a little centralization is enough to provide exponentially smaller 
delay than purely distributed scheduling. We note that, however, none of above work considers a 
game-theoretic framework. 

The supermarket game proposed in this paper falls into a large research area involving equilibrium 
behavior of customers and servers, known as queueing games. A comprehensive survey can be found 
in [[TOl . A particularly relevant paper by Hassin and Haviv [[HI studies a two line queueing system, 
where upon arrival each customer decides whether to purchase the information about which line is 
shorter, or randomly selects one of the lines. It shows how to find a Nash equilibrium and examines 
the externality imposed by an informed customer on the others. 

Finally, the supermarket game is also related to and partially motivated by the theory of mean field 
games in the context of dynamical games [[T2l . [[T3l . The mean field game approach studies a weakly 
coupled, N player game by letting N ^ oo. However, we caution that in the supermarket game, we 
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consider an infinite sequence of customers instead of finitely many customers, which is different from 
an N player game. 

E. Notation and Outline 

Let ^ denote a separable and complete metric space, M.{^) and V{^) be the space of measures 
and space of probability measures on respectively. In this paper, will be £, N, D(M+,N), or 
the space of probability measures on these spaces. Let ^{X) denote the law of a random variable 
X on and 5x denote a point probability measure at x G ^ . Weak convergence of probability 
measures is denoted by 

Define on A^(^) natural duality brackets with L°°(^) as: for (j) e L°°{^) and /i E M{^), 
(0, /i) = / 00?//. Without ambiguity, brackets are also used to denote the quadratic covariation of con- 
tinuous time martingales: let Mi, M2 be two continuous time martingales, then (Mi, M2) is the standard 
quadratic covariation process. Define the total variation norm on A4{,^) as ||/u||tv = sup{(/, /i) : 
ll/lloo < !}• Let ll/ii — /i2||TV be the corresponding total variation distance. For fii, ^2 ^ 'Pi^)^ 
use /ii <st /i2 to denote that fii is first-order stochastically dominated by i-e-^ Xlj^T^A^iO) — 
'l2j=r /^2(j), V/ G C For a; G M, let [x\ denote the maximum integer no larger than x. 

The organization of this paper is as follows. We start with the mean field analysis in Section HIl 
The supermarket game with finite N is considered in Section |llll Section |IV] ends the paper with 
concluding remarks. Miscellaneous details and proofs are in the Appendix. 

IL Supermarket Game in a Mean Field Model 

A. Mean Field Model 

In this subsection, we derive an expression for the expected total cost incurred by a customer in a 
mean field model, by assuming that queue lengths (including customers in service) are independent 
and identically distributed. 

Suppose all the customers except customer i use the mixed strategy fi^i, and let rt{k) denote the 
fraction of queues with at least k customers at time t in the mean field model. Then, the mean field 
equation is given by 

^ = J2 {rlik - 1) - r\{k)) - inik) - n{k + 1)), (3) 

1=1 

which is rigorously derived in Section [nil For now, let us provide some intuition for each of the drift 
terms in ([3]): 



6 



The term \fi^i{l)(rl(k — 1) — r\{k)) corresponds to the arrivals of customers sampling / queues. 
Because queue lengths are i.i.d, r[{k—l) is the probability that the minimum queue length of / uniformly 
sampled queues is greater than or equal to A; — 1. Thus, {r\{k — 1) — ^[{k)) is the probability that the 
minimum queue length of I uniformly sampled queues is /c — 1, which is the same as the probability 
that a customer who samples I queues joins a queue with k — 1 customers. Note that rt{k) is increased 
if a customer joins a queue with k — 1 customers. Therefore, Yla=r ^A*-j(0 ij'ti^ ~ 1) ~ '^t(^)) 
aggregate drift for rt{k) corresponding to arrivals. 

The term {rt{k) — rt{k + 1)) corresponds to departures of customers at queues with exactly k 
customers. 

Since we are interested in the stationary regime, set = to yield equations for the equilibrium 
distribution denoted by r^_j(A;), A; > 0. For A; > 1, 



T, 



iJk - 1) - rj_^(fc) = T,_^{k) - T,_Xk + 1), 



where the random variable L is distributed as ji-i. By summing the above equation for ko < k < +oo 
using telescoping sums and changing kg to k, it follows that 

r^.M = hr,_,{k) = Xu^_,{r,_,{k - 1)), (4) 

where Uf,_,{x) := E^_^ [x^] = Ef=T 

Because queues are independent in the mean field model, 

oo oo 

E[W(L„ = 1 +E[7V(L,)] = 1 + J2nN(L,) > k] = J]rJ:^(fc), 

k=l k=0 

where N{Li) is the length of the shortest queue among the Lj sampled queues. Therefore, 

oo 

C(L,,/x_i) = cJ]r^:^(A;) + c,Li. (5) 

k=0 

Since g{L) = is convex in L e R for a; > and strictly convex for < x < 1, it follows that 

E[W{L, fi_i)] and C{L,ii_i) are strictly convex in L G M. 

Next, we prove several key lemmas which are useful in the sequel. 

Lemma 1. The best response set BR{fj,_i) consists of probability measures concentrated on an integer 
or two consecutive integers. 

Proof: Let //j e BR(/i_j). Suppose there exists Li < L2 < e C such that //^(Li) > and 
IJii{L2) > 0. Then, by the definition of BR, 
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which contradicts the fact that C{L,fi^i) is strictly convex in L and thus the conclusion follows. ■ 

Remark 1. LemmaU] implies that a probability measure fi G BR{fi^i) can be identified with a unique 
real number L G [1, Lmax]- ^ number L G [1, Lmax] with L = [L\ +p for some < p < 1 is identified 
with the probability measure with mass 1 — p at [LJ and p at [LJ + 1. Thus, we use real numbers to 
refer to probability measures in BR{fi^i). 

The next lemma translates the stochastic dominance relations between strategies into the stochastic 
dominance relations between mean field equilibrium distributions. 

Lemma 2. Fix any fii, fi2 ^ 'P{'^) ^^ch that ni <st jj,2- Then, for all /c G N, r^^{k) > r^^{k). 
Furthermore, if Hi <st 1^2, then for all k > 2, r^^{k) > r^^{k). Also, it follows that for all k eN and 
all G V{C), r^{k) < and C{L,fi) is bounded independently of L and ^. 

Proof: Since /ii <st yU2, it follows that for all x G [0, 1], u^^{x) > m^2(x). We prove the lemma 
by induction. If = 0, then r^i(O) = r^^{0) = 1 and r^i(O) > r^^i^) trivially holds. If r^i(A; — 1) > 
r^^{k-l), then 

r^,{k) = Xu^^ [r^^{k - 1)] > Xu^^ [r^^{k - 1)] 
> ^Uf,^ ['"/.al^- 1)] = r^,^{k). 

Therefore, for all k E N, r^-^{k) > r^^{k). Moreover, if /ii <st /i2, it follows that for all x G (0, 1), 

u^^{x) > u^^{x). Thus, for all k > 2, r^^{k) > r^^{k). 

Let yUi = 5i be a point measure at singleton L Then, r^^{k) = A^. Because for any /i G V{C), 
<st /U, r^{k:) < X^ and /i) < c/(l - A) + CsL^-^^. ■ 
The next lemma states that C{jJLi, jji^i) is a continuous function. 

Lemma 3. C{fii,fi-i) is jointly continuous with respect to fii and fi-i. 

Proof: See the proof in Appendix |Al ■ 

B. Existence and Uniqueness of Nash Equilibrium 

In this subsection, we show the existence of a mixed strategy Nash equilibrium. It is easy to see 
that if there exists fi* with fi* G BR(/i*), i.e., fi* is a fixed point of the best response correspondence, 
then is a mixed strategy Nash equilibrium. Thus, it suffices to show the existence of such a fixed 
point. The Kakutani fixed point theorem is used to prove it. 
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(Kakutani's Theorem) Let 5 be a nonempty, compact and convex subset of some Euclidean space 
M". Let (yf : 5 — 7- 2"^ be a set- valued function on S with a closed graph and the property that g{x) is 
a nonempty and convex set for all x E S. Then g has a fixed point. 

In our setting, S = V{C) and g = BR. It is known that V{C), as a space of probability distribution on 
finite set, is a nonempty, compact and convex subset of some Euclidean space. Also, since C{fii, fx^i) 
is continuous, by Weierstrass Theorem, BR(yU_j) is a nonempty set. In addition, C{fii,iJ,-i) is linear 
in Hi, so BR(yU_j) is a convex set. The last and key step is to prove that BR has a closed graph, i.e., 
suppose jj,^^- —7- yU„j, yUj-"-* G BR(/i^,;'') and yUj-"-* — )• fii, we need to show that /ij G BR(/i_,;). It is proved 
in the following theorem using the continuity of C(/ij,/i_j). 

Theorem 1. The supermarket game has a mixed strategy Nash equilibrium in the mean field model. 

Proof: See the proof in Appendix El ■ 
The uniqueness of mixed strategy Nash equilibrium is proved next in case < 1/2. Some definitions 

and lemmas are introduced first. 

For customer z, her marginal value of sampling at an integer Li with all the others adopting 

is defined as 

Intuitively, V{Li, characterizes the reduction of expected waiting time when customer i increases 
the number of sampled queues from Lj to Lj + 1. For ease of notation, define V{Q,pi) = oo and 
V{Lma.x,l^i) = 0. Since E[H^(Lj, is strictly convex in L G M, V{Li,fi_.i) is strictly decreasing in 
Li. The following lemma characterizes the best response using V{Li,p_i). 

Lemma 4. Fix any p^i G V{C) and any integer Li G C Then Li G BR{p-i) if and only if 

V{L,, p_i) < cjc < V{Li - 1, (6) 
Furthermore, fix any non-integer Lj G [1, i^max]- Then Li G BR{H_i) if and only if 

V{[Li\,fi^i) = Cs/c. (7) 

Proof: See the proof in Appendix O ■ 
The next lemma proves a global monotonicity property of V{Li,ix_i) in case < 1/2, which is 
useful for showing the uniqueness of Nash equilibrium in that case. 

Lemma 5. Assume A^ < 1/2 and fix any p^i,p^i G V{C) such that p-i <st P-i- Then V{Li,p_i) > 
V{Li,p_i) for I < Li < Lmax — 1- Furthermore, let Li G BR{fi^i) and Li G BR{jl_i), then Li < Li. 
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Proof: See the proof in Appendix |Dl ■ 

Remark 2. The key ingredient in the proof of Lemma \5\ is to show that .(2) < Li/ {Li + 1), which 
is true for any Li and if < 1/2. The same argument is used in the next subsection. 

Lemma [5] implies that for < 1/2, a customer tends to sample fewer queues when all the other 
customers sample more queues. This is an instance of avoid the crowd behavior IfTOl . leading to 
uniqueness of the Nash equilibrium. 

Theorem 2. If < |, the supermarket game has a unique Nash equilibrium in the mean field model. 

Proof: A Nash equilibrium exists by Theorem [IJ Let L*, L* G [1, Lmax] denote two possible Nash 
equilibria; we show that = L* . 

Without loss of generality, suppose L* < L*\ by Lemma [5l L* > L*, which is a contradiction and 
concludes the proof. ■ 

However, the marginal value of sampling V{Li, L^i) is not always increasing in L_j G [1, Lmax] 
(L_j refers to a probability distribution here) for all A < 1. By numerical results, we find that when 
A = 0.99 and Li = 5, V{Li,L^i) is increasing with L_j for 11 < L_i < 16, as shown in Fig. [U 
This implies that sometimes a customer tends to sample more queues when all the others sample 
more queues. This follow the crowd behavior can lead to multiple Nash equilibria. Such a scenario is 
described in the next subsection. 

C. Computation of Nash Equilibrium and Local Monotonicity Condition 

In this subsection, we show how to find a Nash equilibrium and establish the uniqueness of Nash 
equilibrium if a local monotonicity condition is satisfied. Then, a specific example where multiple 
Nash equilibria exist is constructed. 

The following lemma determines a Nash equilibrium using the marginal value of sampling. 

Lemma 6. Suppose L* is determined by the following procedure: 

(a) If V^(Lmax - 1, ^max) > ^, sct L* = L^ax- Otherwise 

(b) Let L := mm{L G C : V{L, L + 1) < 
(bl) ifV{L,L) < ^, set L* = L. 

(b2) if V{L, L) > ^, set L* = L + q*, where Q < q* < I is the solution of V{L, L + q*) = ^. 
Then, L* is a Nash equilibrium for the supermarket game in the mean field model. 

Proof: See the proof in Appendix El ■ 
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Figure 1. The marginal value of sampling V{5, L) with increasing L from 10 to 20 and varying arrival rate A. 

Next, we introduce a local monotonicity condition and show the uniqueness of Nash equilibrium 
for all values of c and Cs if and only if the local monotonicity condition is satisfied. 

Definition 1. Given < A < 1 and any integer Lmax > 1. the local monotonicity condition is 
satisfied for (A, L^ax) if V{L, L + q) is strictly decreasing over < g < 1 for each integer L with 

1 ^ ^ Liaax ~ 1- 

Theorem 3. Given < A < 1 and any integer Lmax > 1. the supermarket game in the mean field 
model has a unique Nash equilibrium for all values of c and Cg , if and only if the local monotonicity 
condition is satisfied for (A,Lmax)- 

Proof: See the proof in Appendix IB ■ 
The following numerical results, depicted in Fig. [21 show that when A = 0.99, L + q) is indeed 
strictly decreasing with respect to < g < 1 for L = 1, . . . , 9. For L > 10, 

rL+,(2) < ri(2) < no{2) = 0.99'^^^'^^^'^^^-^^ < 10/11 < L/{L + 1), 

which implies that V{L, L + q) is strictly decreasing over < g < 1, in view of the proof for Lemma 
[51 Therefore, for A = 0.99 and any integer L^ax > 1j there exists a unique Nash equilibrium for all 
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Figure 2. For A = 0.99, the marginal value of sampling V{L, L + q) with increasing q from to 1 and varying L. 



values of c and Cg. However, when A = 0.999 and Lmax = 25, V{L, L + q) is strictly increasing in q 
for L = 18, . . . , 24. Therefore, multiple Nash equilibria exist if c^/c = 0.0148, as depicted in Fig. [31 
We see that L = 19, . . . , 24 are pure strategy Nash equilibria and mixed strategy Nash equilibria exist 
between each two consecutive pure strategy Nash equilibria. 



D. Special Case: Two Choices 

In this subsection, we consider a simple special case where all the customers only have two choices, 
1 or 2. 

Fixing a customer i, if L_j = 1 + g, i.e., all the other customers choose one queue to sample with 
probability 1 — q and two queues with probability q, then the stationary distribution in the mean field 
model can be derived as 

r,(0) = 1, rg{k) = \u,{r,{k - 1)), A; > 1, 

where Uq{x) = x^q + x{l — q). Note that this stationary distribution has also been derived in Section 
4.4.1 of ttH. 
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Figure 3. The best response with respect to L for X = 0.999 and ^ = 0.0148. 



Then, the total expected cost of customer i choosing 1 + p is given by 

oo oo 

C{l+p,l + q) = (l-p)(c^rg(A;) + c,)+p(c^rJ(A;) + 2c,). 

fc=0 k=0 

The marginal value of sampling of customer z at 1 is given by 

oo 
k=0 

It follows that the best response under L_j = 1 + g is given by 

1 if c,/c> + 

BR(l + g)=i 2 ifCs/c<V{l,l + q), (8) 
[1,2] if c,/c=\/(l,l + g). 

V 

By numerical results depicted in Fig. HI we find that 1^(1, 1 + q) is strictly decreasing in q for a 
sequence of A up to 0.999. This is strong numerical evidence that the local monotonicity condition is 
satisfied for all A < 1. Therefore, by Theorem[3]and LemmaHl we conjecture that: (i) if 1) < Cs/c, 
then L* = 1 is the unique Nash equilibrium; (ii) if V{1,2) > Cg/c, then = 2 is the unique Nash 
equilibrium; (iii) otherwise, there exists ap* G (0, 1) such that V{1, l+p*) = Cg/c, and thus L* = 1+p* 
is the unique mixed strategy Nash equilibrium. 
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Figure 4. The natural logarithm of V{1, 1 + q) with increasing q from to 1, for varying arrival rate A. 



E. Externality and Social Optimum 

The action of sampling more queues by some customers has an effect on the waiting time of others. 
This effect is called the externality associated with the action and the externality is positive if the 
action reduces the mean waiting time of the other customers. In this subsection, the externality in the 
mean field model is analyzed. It is not clear whether the externality is positive at first sight. On one 
hand, choosing a large number of queues to sample helps a customer find a less loaded queue and 
hence reduces future arrivals' opportunity. On the other hand, it also leads to a well balanced system 
and reduces the average waiting time. 

The following corollary of Lemma [2] implies that the action of sampling more queues by some 
customers has a positive externality on the other customers in the mean field model. To see it, suppose 
in system 1, all the customers adopt a strategy fii, while in system 2, a fraction < p < 1 of them 
samples more queues, i.e., adopts a new strategy /is with /ii <st yUs and all the others still adopt the 
strategy fXi. For system 2, it is equivalent to assume that all the customers adopt a strategy fi2 with 
fi2 = J5/i3 + (1 — It follows that Hi <st 1^2- By Corollary [H system 2 has smaller mean waiting 
time. 
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Corollary 1. If fii,fi2 G V{C) with fii <st fi2, then for all L e C, E[W{L,fii)] < E[W{L,fi2)]. 

Proof: Because /ii <st /U2, by Lemma [21 r^^{k) > r^^{k),'ik > 2. Hence, the conclusion follows 
by invoking the definition of E[H^(L, //)]. ■ 

Remark 3. However, it is unknown whether the action of sampling more queues by some customers 
has a positive externality for finite N case. See Section I///- CI 

Next, we analyze the social optimum, i.e, the minimum of total cost of all the customers. Suppose 
all the customers use the mixed strategy yu; the expected total cost per unit time Csum(Ai) is given by 

Csum(/u) = AC(^,/i) = A ^ = A ^ /i(L)(^r^(A;) + c,L). 

L=l L=l fc=0 

Therefore, a minimizer of the following minimization problem: 
is a social optimum. 

Lemma 7. The social optimum is a probability measure concentrated on either an integer or two 
consecutive integers. 

Proof: See the proof in Appendix [Gl ■ 

Theorem 4. No Nash equilibrium ^* can strictly stochastically dominate the social optima fi*^^ in the 
mean field model. 

Proof: The total cost can be decomposed into two terms as 

-^'max -^Tnax -^^max 

f^iL)C{L, fi)=J2 l^iL)C{L, /i^) + J2 l^iL) {C{L, /i) - C{L, /i*)) . 

L=l L=l L=l 

Suppose the Nash equilibrium pL* >st /igoc then by the positive externality result, we have 



Also, by definition of /x* 



Therefore, 



C(L,^soc)-C(L,^*) >0. 



L=l L=l 



^ /isoc(i^)C(L,/isoc) > Y n''{L)C{L,ij,*), 

L=l L=l 
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which is a contradiction to the definition of fisoc- ■ 

Remark 4. Since Nash equilibrium L* and social optimum fi*^^ can be identified with real numbers 
in [l,Lmax], the above lemma further implies that L* < L*^^, i.e., no Nash equilibrium can be above 
the social optimum. 

F. Heterogeneous Waiting Cost 

In this subsection, the heterogeneous waiting cost is considered. In particular, assume that there is 
a nondegenerate continuous probability density function for waiting cost c, i.e., /J^™'"' f{c)dc = 1. 

Fix a customer i, let Li{-) denote a function from [0,Cmax] to C We call Li(-) the strategy of 
customer i. In particular, if customer i has a waiting cost c, then she chooses -Lj(c) queues to sample 
uniformly at random. Now suppose all the other customers use L^i(-), the expected total cost for 
customer i is given by 

= cE[WiLi{c), + c,L,(c). (9) 

The goal of customer i is to minimize the expected total cost by choosing the optimal Li(-). Define 
the best response correspondence as BR(L_j(-)) := arg minj^^(.) C(Lj, 

If Li{-) E BR(L_i(-)), then Li(-) must be a nondecreasing step function with respect to c as depicted 
in Fig. [51 which is proved in the following lemma. 

Lemma 8. Suppose Li{-) G BR{L^i{-)), then Li{-) is a nondecreasing step function in c. 
Proof: Suppose < c < c < Cmax, since Lj(-) G BR(L_j(-)), we have 

cE[W{L,{c),L,,)]+cMc) < cE[W{L,{c),L_i)]+cMc), 
cE[W{L,{c),L_,)]+CsL,{c) < cE[W{L,{c),L.i)]+CsLi{c). 

Adding the above two inequalities up, we get 

(c - c){E[W{Li{c), - nWiL,{c), < 0. 

Therefore E[W{Li{c), L_i)] < E[W{Li{c), L_i)] and thus L,(c) > Li{c). U 
Define the strategy space S as the collection of all possible nondecreasing step functions from 
[0,Cmax] to C. By Lemma [HI it suffices to consider S for finding pure strategy Nash equilibrium. 
There is a bijective mapping between the strategy space S and probability space V{C). In particular, 
suppose L(-) G S is given, and let = cq < ci < ■ • • < cl^^^ = Cmax denote the jumping points. 
Then, define = /^^' ^ f{c)dc. On the other hand, suppose /i G 'P(>C) is given, then the equation 
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Figure 5. The best response strategy Li{-) £ BR(L_i(-)) is a non-decreasing step function. The jumping points are denoted by 

= Co < Cl < ■ ■ ■ < Cr^^^ = Cmax- 



= ^ f{c)dc is solved to get the unique jumping points = cq < ci < ■ ■ ■ < cl^^^ = Cmax- 
Thus, L(-) can be constructed as L(c) = / for q_i < c < q. Define a metric on the strategy space 
S as c/(Li(-), L2(-)) := — I^L2\\- Also, Let F denote the bijective mapping from L(-) to hl and 
F^^ denote the inverse mapping. It is easy to see that F and F^^ are continuous. 

Next, we show the existence of a pure strategy Nash equilibrium in the mean field model. First, 
let us derive the expression of E[14^(Lj, in mean field equilibrium. Suppose all the customers 
except customer i use the strategy L_i(-). Due to the bijective mapping between the strategy space 
and probability space, it is equivalent to consider the case in which all the customers use the mixed 
strategy fXL^i- Therefore, the mean field equilibrium distribution satisfies rL_^{k) = r^^ (k), and the 
expected waiting time of customer i using strategy Lj(-) is given by 

oo 
fc=0 

Lemma 9. The best response correspondence BR{L^i) is a continuous function. 

Proof: See the proof in Appendix |Hl ■ 
Define the best response from V{C) to V^C) as BR(yU_j) := F(BR(F^^(/i_i))). Then the existence 
of a pure strategy Nash equilibrium is equivalent to the existence of a fixed point of BR. The Brouwer 
fixed point theorem is used to prove it. 




Figure 6. The expected waiting cost of customer i choosing L while the others choosing a fixed L_i. In this example, Lmax = 5 and 
ci, . . . , C4 are the jumping points of Li{-) G BR(L_,;(-)). 



(Brouwer's Theorem) Every continuous function g from a convex compact subset of a Euclidean 
space to itself has a fixed point. 

Theorem 5. The supermarket game with heterogeneous waiting cost has a pure strategy Nash equi- 
librium in the mean field model. 

Proof: Note that BR is a continuous function because F, F^, and BR are continuous functions. 
Also, V{C) is a convex compact subset of an Euclidean space. Therefore, by Brouwer's fixed point 
theorem, BR has a fixed point fi* and thus F^^^jj*) is a pure strategy Nash equilibrium. ■ 
In the sequel, the stochastic ordering between two possible pure strategy Nash equilibria is analyzed. 
For customer i, her marginal value of sampling at an integer Lj(c) with all the others adopting L_i{-) 
is given by 

oo 
fc=0 

The next lemma generalizes Lemma [5] and proves a global monotonicity result of V(Lj(c), 
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Lemma 10. Assume < 1/2 and fix any L^i{-), L^i{-) G S such that fii - <st fJ'i^i- Then 
V{Li{c),L_i) > V{Li{c),L_i) for all 1 < Li{c) < L^ax - 1- Furthermore, let Li{-) G BR{L_i{-)) 
and Li{-) G BR{L_i{-)), then /x^^ <st /i^,. 

Proof: By Lemma[5j it follows that V{Li{c), L^i) > V{Li{c), L_i) for all 1 < Li{c) < L^ax - 1- 
Next, we show that <st hl.. Denote the jumping points of Lj and Lj by = cq < Ci < • • ■ < 
CLmax = Cmax and = Co < Ci < • ■ ■ < cl^^^ = Cmax respectively. It suffices to show that Cj < Cj for 
< j < /^max- Fig. [6] shows that for 1 < j < L^ax - 1, 

_ Cq 



which implies that cj < Cj. ■ 

Corollary 2. Let L*{-) and -C/2(') denote two possible distinct Nash equilibria for supermarket game 
with heterogeneous waiting cost in the mean field model. If \^ < \, then fi^* and fii* cannot be 
stochastically ordered. 

Proof: Suppose fi^* <st /ii*. Then, by Lemma [TOl fii* >st ^j-l*, which is a contradiction to 
assumption and concludes the proof. ■ 

Remark 5. We are unable to prove the uniqueness of pure strategy Nash equilibrium because stochastic 
dominance is not a total order, i.e., there exists fii and ^2 which cannot be stochastically ordered. 

III. Supermarket Game for finite 

In this section, the supermarket game for finite is considered. We first show that the mean field 
model is the right limit as — )■ 00, which further implies that a Nash equilibrium in the mean field 
model is an e-Nash equilibrium in the finite model. Then, the externality of sampling more queues 
by some customers in the finite A^ model is studied. 

A. Propagation of Chaos 

In this subsection, we rigorously prove the propagation of chaos and coupling result for the finite 
A^ model with all the customers using strategy ji G V{C). Our proof techniques are essentially the 
same as [IJ and [2J. The only difference is that we consider a slightly more general case where the 
customers' choices are random instead of deterministic and fixed. 

Denote by Qf [t] the length of queue i at time t. The process of {Q^} is Markov, and the empirical 
distribution = (l/N) Sqn has samples in P(D(M_|_, N)). Define the marginal process = 
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Thm. 7 



N ^oo 
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t ^co 



Thm. 10 
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t ^co 



(In equilibrium) — ^ (Fixed point) 



Figure 7. The roadmap to show a chaoticity result in equilibrium. 



{Qt^)t>o as 

1 ^ 

1=1 

The marginal process has sample paths in ro(R_|_, V(N)). The fraction of queues of length at least 
k at time t can be written as r^{k) = Q^{[k, oo)). 

For r G V{^), a sequence of random variables (Xj)i<j<7v on is T-chaotic if for any fixed 
integer / > 1, as — oo, 

j^(Xi,...,xo^r®'. 

A sequence of random variables (Xj)i<j<Ar on is exchangeable if for any permutation vr : 

[l,...,iV]^[l,...,iV], 

The proof roadmap is summarized in Fig. |71 We first prove a chaoticity result on path space (Thm. 
7), i.e., there exists a F G P(D(M+,N)) such that {Qf)i<i<N is F-chaotic, if the initial condition 
{Qf {^))i<i<N is chaotic. Then, we show a chaoticity result in equilibrium (Thm. 10) by (i) taking the 
large N limit and proving that the solution of the mean field equation converges as t — > oo to a fixed 
point (Lemma 13); (ii) taking the large t limit and proving the finite N model is ergodic (Thm. 9); 
(iii) using the chaoticity result on path space (Thm. 7) to finish the proof. The coupling result proved 
in Theorem 8 is used to show the ergodicity result in Theorem 9. The chaoticity result on path space 
is proved using Proposition 2.2 in [15] and the nonlinear martingale problem approach. First, some 
useful definitions and preliminary lemmas are stated. 
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Let {n)k := n{n — 1) ■ ■ ■ {n — k + 1) for integer n > k > 0. Define the A; -body empirical measure as 



N 



ji,...,ifc=l, distinct 



1 

k,N ^ . ^ 

ii ,...,ifc=l,distinct,^i 



and the fc-body empirical measure for queues other than i 

N 

-IK. 

ll,...,lk 

with their marginal process Q'''^ and Q^'^ . The Lemma 3.1 in [HI, restated in the following lemma, 
proves that the /c-body empirical measure is close to -product of the empirical measure z/^. 

Lemma 11. We have ||z/'='^ - (z/^)®^||tv = 0{1/N) and ||z/f'^ - = 0{1/N). 

For a bounded function </> on N, set = 0(x + 1) — and (p^{x) = (j){x — 1) — 4>{x). Let 

XL be a function as 

XL : . . . , Xl) G — l^^^Ji^ g {0, 1/L, . . . , 1/2, 1}. 

The next lemma gives a martingale process induced from Markov process {Qf}- 
Lemma 12. We have a martingale process M*^'*'^ defined by 

ft ^max 

Mf =0(Qf (t)) - 0(Qf (0)) - / i^{L){\L{xL{Qns), ■), Qf-''^(^))0nQf (^)) 

"^0 L=l 

+ lQN^,)^,ct>'{Xf{s))}ds + 6^'^'^(t) (10) 

and e'^'*'^(t) = t||0||ooO(l/A^) uniformly. The quadratic covariations (M'^''"'^ , M^'^'^) are zero for 
i ^ J and E[(M'^'*'^, M<^'^'^)t] < C\\(j)\\l^t^ for a constant C depending on t. 

Proof: The martingale statement is obtained from the Dynkin formula in stochastic analysis. 
The bound on e'^'^'^it) follows from Lemma [HJ Also, because there are no simultaneous jumps, the 
quadratic covariations (ikf^'*'^, M'^'^'^) are for i 7^ j. Lastly, by the definition of quadratic variation 
and M/'^'^, 

E[(M<^'^'^, Mt>^^^%] = E[(Mf < 3 (211011^ + ((AL^,. + l)||0||oot)') < CU\tt\ 



L=l 
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Next, we introduce the nonlinear martingale problem which is useful to prove the propagation of 
chaos result. We say a law V E P(D(M+,N)) solves the nonlinear martingale problem if for any 
bounded function on N, 

r.* Ln 



t) - </^Wo 



f*t -'-'max 

/ KL)WxLiQs, ■), rf -')0+(g.) + iQ^>i<p-iQs)}ds (11) 

-^0 L=l 

defines a F-martingale. It solves the martingale problem starting at 7 if Fq = 7. 
By taking the expectation over two sides of (fTT)) and using the result 

L(xL(x)0+(xi),rf (rfx)) = (0+(min{x}),rf (rfx)), 

we get the nonlinear Kolmogorov equation. We say a (deterministic) process {Tt)t>Q E D(]R_|_, P(N)) 
solves the nonlinear Kolmogorov equation if for any bounded function on N, 

pt ^max 

(0,ri) = (0,ro)+ / 5^/i(L){A(0+(min{x}),rf^(rfx)) + (l.>ir,r.)}rfs (12) 

and it solves the equation starting at 7 if moreover Fq = 7. Taking equal to l[fc,oo), and using the 
fact that 

(0+(min{x}), rf (rfx)) = <PH^) (rf([x, 00)) - ([x + 1, 00))) , 
we obtain the mean field equation for rt{k) = Tt{[k, 00)) as 



dt 



X Y l^iL) {r^k - 1) - rf (A;)) - {n{k) - n{k + 1)). (13) 

L=l 

The following theorem, which is essentially the same as Theorem 3.3 in [|T]|, proves that the nonlinear 
martingale problem has a unique solution. 

Theorem 6. Let 7 E V{N). Then there is a unique solution T in P(D(M_|_,N)) for the nonlinear 
martingale problem 477]) starting at 7. Its marginal process {Tt)t>Q is the unique solution for the 
nonlinear Kolmogorov equation 472]) starting at 7, and r = {rt)t>o defined by rt{k) = Tt{[k, 00)) is 
the unique solution for the mean field equation 473]) starting at tq. 



Proof: See the proof in Appendix H] ■ 
The following theorem proves a propagation of chaos result on the path space. 

Theorem 7. Assume that {Qq )n>i converges in law to a -y in V(N), or that (r^)Ar>i converges 
in law to Tq with rQ^k) = 7([A;, 00)). Then {i'^)n>i converges in probability to the unique solution 
r for the nonlinear martingale problem 4771) starting at 7, and if (<5f'(0))i<j<Ar is exchangeable 
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then {Qf)i<i<N is T-chaotic. Moreover, {Q'^)n>i converges in probability to (Tt)t>o <^nd {r-'^)N>i 
converges in probability to {rt)t>Q,for uniform convergence on bounded intervals over t, where rt{k) = 
Ttdk, oo)). Note that {Tt)t>o is the unique solution for the Kolmogorov equation ([72]) starting at 7, 
and {rt)t>f) is the unique solution for the mean field equation ([721) starting at Tq. 

Proof: See the proof in Appendix |Jl ■ 
The following lemma proves that in the mean field model, starting with any appropriate initial 
distribution, the solution of mean field equation ([T3] ) will converge to its fixed point. 

Lemma 13. In the mean field model, starting with any initial distribution rQ{k) such that X]fc>o '"o(^) < 
00, the solution of mean field equation ([721) rt{k) converges in t to a fixed point r{k) given by 

r{<d) = l,r{k) = \u^{r{k-l)). (14) 

Proof: See the proof in Appendix [Kl ■ 
Let r[^'^\k) and Qf^'^\t) denote the fraction of queues with at least k customers and the number 
of customers at queue i respectively for system j at time t. We can prove the following coupling result 
in the similar way as Turner did for the fixed number of sampling queues [|2J|. 

Theorem 8. Suppose all the customers adopt fii in system 1 and ji2 in system 2 with ni <st Ii2- Then 
there is a coupling between the two systems such that for all t and x, 

i i 

where [a]+ = max{a, 0}. It follows that for any nondecreasing convex function h and for all t, 

i i 

Proof: A coupling between fii and 112 can be constructed as follows, let Fi and F2 be the 
cumulative distribution function for /ii and /i2 respectively. Let U denote the uniform distribution 
over [0,1]. Then it follows that F{'^{U) and F2^{U) is distributed as fii and 112 respectively. Also, 

F{\U) < F,~\U). 

Let us couple the two systems as follows. All arrival times and departure times are the same in both 
systems, except that departures that occur from an empty queue are lost. At the times of departure, 
we arrange the queues in each system in order of queue lengths, and let a departure occur from the 
corresponding queue in each system; for example, if it occurs from the longest queue in system 1, 
then let it also occur from the longest queue in system 2. At arrival times, the customer in system 1 
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samples ^(f/) queues; while the customer in system 2 first chooses the same set of queues chosen 

by system 1 and then samples additional iF2'^{U) — F^^{U)) queues. 

Using Theorem 4 in [i2| and the fact that F^^{U) < F2^{U), the conclusion follows. ■ 
The coupling result is used to show the ergodicity for Qf, which is proved in the following theorem. 

Theorem 9. For N > 1, {Qf)i<i<N is ergodicfor A < 1, and thus has a unique stationary distribution. 
Furthermore, in equilibrium {Qf)i<i<N is exchangeable. 

Proof: The proof is the same as that in Theorem 4.2 in [[T]|. ■ 

The following theorem proves the chaoticity in equilibrium. 

Theorem 10. Let be defined as 47?1) and define 7^ G V(N) as '~i^{k) = r^{k) — r^{k + 1). In 
equilibrium, {Qf)i<i<N is T -chaotic, where T is the unique solution for the nonlinear martingale 
problem 4771) starting at 7^, and Q (the Markov process with measure T) is in equilibrium under T. 
Hence in equilibrium {Q^)n>i converges in probability to the constant process identically equaling 
to 7^ for all t and {r^)N>i converges in probability to the constant process identically equaling to 
for all t, for uniform convergence on bounded intervals. 

Proof: See the proof in Appendix O ■ 

Remark 6. Let Qf{0) be the length of queue i in equilibrium. Theorem [70l implies that {Qf {0))i<i<N 
is 'y ^-chaotic. By the definition of chaoticity, it follows that the joint equilibrium distribution of any 
fixed number of queues converges to a product distribution, i.e., for any fixed integer I > 1, as N ^ 00, 

^(QS^)(0),...,Q;^)(0))^7r. 

B. e-Nash Equilibrium for Finite N 

In this subsection, the e-Nash equilibrium for finite queues is considered. For a fixed customer 
i, suppose she uses the mixed strategy /ij, and all the other customers use fi-i. Let W^^\fii, ^-i) 
and C^^\jj,i, fi^i) denote her waiting time and total average cost for queues. Then, the expected 
waiting time can be derived as 

E[iy(^)(L„ =E[min{QW(l),...,Q(^)(L,)}] +1, 

where Q^^\i) is the length of the ith sampled queue. 

Next, the definition of e-Nash equilibrium in the finite A^ model is introduced. 
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Definition 2. We call fi* G V{C) an e-Nash equilibrium in the finite N model, if for any e > 0, with 
N sufficiently large, 

C^^\fi\fi*) < C^^\fi,fi*) + e, for all fi G V{C). 

Theorem 11. Let fi* be a Nash equilibrium for the supermarket game in the mean field model, then 
is an e-Nash equilibrium in the finite N model. 

Proof: Let Z denote the random variable associated with the mean field equilibrium distribution 
f^i_Xk)- By Theorem [TOl we have that for any fixed Lj, as — t- oo, 

^(Q(^)(l), . . . , Q(^)(L,)) =^ 

By the coupling result in Theorem [8l 

E,„^ [min{gW(l),...,QW(L,)}] < E,_^(gW(l)) < E,,(gW(l)) = 

and hence W''^\Li, is uniformly integrable. Therefore, for Ve > 0, there exists iVo G N such that 
when > A^o, 

|E[iyW (L„ -E[iy(L„/i„,)] I < ^, 

and thus 

<e/2, 

where W{Li, jjL^i) and C{fj.,ix-i) are the waiting time and total average cost respectively in the mean 
field model. By definition of fi*, 

C{iJ.\iJ,*) < C^^,^*), for all /i G V{C). 

Then, it follows that 

C(^)(/i^/i^) < C(^)(/x,/i*) + e, for all ^l G V{C). 
Therefore, /x* is an e-Nash equilibrium in the finite system. ■ 

C. Externality for finite N 

In this subsection, we study the externality of sampling more queues by some customers in the 
finite A^ model. 

Theorem [8] implies that the action of sampling more queues by some customers has a nonnegative 
externality on the other customers who only sample one queue. To see it, suppose all the customers 
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adopt /ii in system 1 and jj,2 in system 2 with /ii <st /i2- Since E[iy(l,/ij)] = '^[j^^jQ[^\j)] + 1 
for i = 1,2, it follows from Theorem [8] that E[Vr(l,/ii)] > E[W {1,1^2)]- 

However, it is unknown whether sampling more queues by some customers has a positive externality 
on the other customers who sample more than one queue, because we are not able to prove that 
E[iy(L,yUi)] > E[11/(L, /i2)] for L > 2 simply by the coupling result. Consider a special case where 
Hi = 5i, fi2 = §2 and N = 2. For fii = 5i, due to independence among queue 1 and 2, E[iy(2, /ii)] = 
However, for fi2 = 62, queue 1 and 2 are correlated and we have no explicit expression for 
E[Vr(2, (12)] - For the heavy traffic limit A — t- 1, Kingman showed that E[H^(2, fi2)] ~ j^p- in Theorem 
6 in [fT6l . Thus, the externality is approximately in the heavy traffic limit for this special case. 

IV. Conclusions 

Our results indicate that the equilibrium picture for the supermarket game can be somewhat com- 
plicated, at least if 0.5 < < 1. In particular, there may be multiple Nash equilibria, stemming from 
the fact that customers do not always have an "avoid the crowd" response as when A^ < 0.5. However, 
this complication seems to occur only for A close to one and L^ax fairly large. Also, at least in the 
mean field model, the positive externality of increased sampling holds for the whole range A < 1. 

Appendix 

A. Proof of Lemma \3\ 

First, let us show that C{jj,i, jj^^i) is continuous in /Xj for any fixed Let — /UjHtv — !■ as 
n — 7- 00, then we have 

Li = l 

< if 2^ I fi^^-'iLi) - 1^ 0, as n ^ 00, 

Li=l 

where (a) follows from the fact that C{Li,ix_i) is uniformly bounded, which is proved in Lemma [2l 
Furthermore, V{C) is a compact set and thus C is uniformly continuous in /ij for any fixed fi^i. 

Second, we show that C is continuous in for any fixed /ij. It suffices to show that E[iy (L, fi-i)] 
is continuous in Let WjJ,^^] — /i-illxv — )■ as n — t- 00. First, note that the mean field equilibrium 
distribution r^_.{k) is continuous in for each k. Second, by Lemma|2l rij,{k) < A'^, V/c, V/x G V{C). 
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Therefore, for Ve > 0, by choosing sufficiently large K, 

K 



k=0 
K 



fc=0 



Also, since rfj,_^{k) is continuous in n_i for each k, by choosing sufficiently large Nq, it follows that 
when n> Nq, 



A;=0 ' fc=0 



Therefore, 

|E[l^(L,/i5)]-E[W^(L,^_,)]|<e. 

Furthermore, V{C) is a compact set and thus C is uniformly continuous in fi^i for any fixed /Zj. 

Lastly, let us show that C(/ij, is jointly continuous with respect to fii and Let /i^^) — )• 
{Hi, H-i) in total variation distance as n — )■ oo, then 

=1 Y^ciu,^^^::^)[^^t'\L^)-^^i^iU))\ + \c{^^.,^^^::})-Ci^^.,^..)\ 

Li=l 

Lmax 

— > 0, as n — > oo, 
which concludes the proof. 

B. Proof of Theorem [7] 

Since /if ^ G BR(/i5), for all (i, e V{C), 

C(/i.,/i5)>C(/xS"\/i5). (15) 

By the continuity of C proved in Lemma [31 

lim C{p,i,jj,'-^}) = C{fli,iJ,-i), 

n— >oo 

lim = C{iJ,i,iJ,_,). 

n— >oo 
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Therefore, for all fii G V{C), C{fii, fi-i) > C{fii, fi-i) and thus /Xj G BR(yU_j). Then, by the Kakutani's 
Theorem, there must exist a /i* as a fixed point of BR and thus /i* is a mixed strategy Nash equilibrium. 
Furthermore, by Lemma [TJ /i* can be identified with a real number L* E [1, i^max]- 

C. Proof of Lemma |4] 

The first half of the lemma is proved first. For any integer Li E C such that Li E BR(yU_j), 
C{Li,n^i) < C{Li — and C{Li,fi_i) < C{Li + Thus equation Q easily follows by 

invoking the definition of C{Li,n^i). 

Now suppose equation Q holds. Since V{Li,^_i) is strictly decreasing in Li, 



I < Cs/c if L > Li, 
which implies that for all L E C, C{Li,fi^i) < C{L,ii^i) and thus E BR(/i_j). 

The second half of the lemma is proved next. For any non-integer Li E [1, Lmax] such that Li E 
BR(/i„i), [Li\ E BR(;U_i) and [Li\ +1 E BR(/i_i). It follows from equation ^ that 



which implies that [LjJ E BR(|U_i). Similarly, we can show that [LJ + 1 E BR(|U_j). Therefore, 
Li E BR(/i_i). 

D. Proof of Lemma |5] 

We first show that V{Li,^_i) > V{Li,pL_i) for all 1 < Lj < Lmax — 1, i.e., 




V{ [Li\ , li.i) < cjc < V{ [L,J +1-1, fi.i), 



(16) 



which implies equationdV]). 

Now suppose equation ([7]) holds. Since V{Li,n_i) is strictly decreasing in L, 



V{[L,\,fi_i) = Cs/c < V{[Li\ - l,/i_,) 



(17) 



oo 



oo 



k=0 k=0 



We prove a stronger conclusion, that is, for k > 2, 




By Lemma H for > 2, 



rUk)<rUk)<^'<l- 
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Define function g(x) = x^^ — x^^^^. It is easy to calculate that g'{x) = x^^~^{Li — (Li + l)x). It 
follows that for x < |, g'{x) > 0. Therefore, g{r^_-{k)) > g(rfi_.{k)),\/k > 2. 
Next, we show that < L^. 

Case 1 (Li is an integer): Since Li E BR(/2j), it follows from Lemma |4] that 

V{Li - l,fi_i) > V{Li - l,fi_i) > cjc, 

which implies that [LJ > Lj — 1 again by Lemma |4] and thus Li > L^. 
Case 2 (Lj is a non-integer): From Lemma |4l 

V{[U\,ii_i) > V{[Li\,^_i) = Cs/c. 

By Lemma |4l it follows that: (i) if Li is a non-integer, then [LjJ > [Lj J ; (ii) if Li is an integer, then 
Li > [Li\. Therefore, Li > Li. 

E. Proof of Lemma |6| 

For case (a), F(Linax — 1,-^max) > T-- By Lemma H it follows that Lmax G BR(Ljnax) and thus 
Lmax is a Nash equilibrium. 

As for case (b), note that L is well defined, because V^(Linax — l7-^max) < In the case (bl), 
y(L,L) < ^ and by the definition of L, ^(L - 1,L) > ^. Therefore, by Lemma H L e BR(L). 
In the case (b2), V{L,L) > ^ and by the definition of L, V{L,L + 1) < ^. Since V{Li,L_i) is 
continuous in L_j G M, it follows that there exists a q* with < < 1 such that V{L, L + q*) = ^. 
Therefore, by Lemma H L + G BR(L + q*) and thus L + g* is a Nash equilibrium. 

F. Proof of Theorem \3\ 

(Proof that the local monotonicity condition is sufficient) Let G [1, ivmax] denote any Nash equilib- 
rium, and define L := min{L G C : V{L, L + 1) < Cs/c}. By definition, V{L^i^^, Lmax + 1) = and 
thus L is well defined. Also, it follows that 

V{L, L + l)> V{L + 1, L + 1) > V{L + 1, L + 2), for 1 < L < L^^x - 2, 

where the first inequality follows from the fact that V{Li,n_i) is strictly decreasing in , and the 
second inequality follows from the monotonicity assumption of V{L, L+q) in q. Therefore, V{L, L+1) 
is strictly decreasing in L. 

Next, the following three different cases of L are considered: 
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Case 1 (1/ = Lmax): By the definition of L, it follows that V{L,L + 1) > c^/c for all 1 < L < 
Linax - 1- Hence, 

V{L, L) > V{L, L + l)> Cs/c, for 1 < L < L^^ax - 1- (18) 

Therefore, by Lemma IH L* cannot be an integer less than Lmax- Now if L* is a non-integer less than 
Lms^x, then [L*\ < L^^^ — 1. Thus, it follows from (fTSl ) that 

V{ J , V) > V{ [L*\ , [L*\ + 1) > Cs/c. (19) 

On the other hand, by the definition of L* and LemmalU V{ [L*\ , L*) = Cg/c, which is a contradiction 
to (fT9l ). Therefore, L* must be Lmax- 

Case 2 (L < Lmax — 1): Since V{L, L + 1) is strictly decreasing in L, by the definition of L, 
V{Lma.yi — l,Lmax) < Cg/c. Therefore, by Lemma |4l L* cannot be Lmax- 

Case 2(a): If L* is an integer less than L^^,^, then 

V{L\ L*) < cjc < V{L* - 1, L*). 
By monotonicity assumption, it follows that 

V{L, L + l)> V{L* - 1, L*) > cjc for L < - I, V{L\ + 1) < V{L\ L") < cjc. 

Therefore, L* — 1 < L < L* and thus L* = L. 

Case 2(b): If L* is a non-integer, then V{[L*\,L*) = Cg/c. By monotonicity assumption, 

V{[L*\, [L*\+l) < V{[L*\,L*) = ^. (20) 

c 

Also, for L < [L*\ - 1 , 

ViL,L + l) > V{[L*\ - 1, [L*\) > V{[L^\, LL^J) > V{[L''\,L*) = cjc. 

Therefore, [L*\ - 1 < L < [L*\ and thus [L^^J = L. Furthermore, since ¥{[1*1, [L^l + 1) < 
Cs/c < V{[L*\, [L*\), by monotonicity assumption, there is a unique q* with < < 1 such that 
V{L, L + q*) = ^ and thus L* = L + q" . 

(Proof that the local monotonicity condition is necessary) Suppose the local monotonicity condition 
is not satisfied. It is well known that a continuous, injective (i.e. one-to-one) function g on an interval 
[a, h] is either strictly monotone increasing or strictly monotone decreasing, so there are two cases to 
consider. 

Case 1: There exists a L with 1 < L < L^^ax such that V{L,L + q) is strictly increasing for 
< g < 1. Because V{L — 1, L) > V{L, L), c and Cg can be selected to satisfy 

V{L, L) < cjc = V{L, L + qo)< V{L - 1, L) 
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for some < go < 1- From Lemma IH it follows that L is a pure strategy Nash equilibrium and 
L + go is a mixed strategy Nash equilibrium. Therefore, the supermarket game has at least two Nash 
equilibria. 

Case 2: There exists a L with 1 < L < Ljaia such that V{L, L + = V{L, L + q-^) with 
< g2 < gs < 1- Then, if c and Cg are selected so that ^ = V{L, L + q2) = V{L, L + ga), both L + q2 
and L + q^ are Nash equilibria. 

Therefore, in either case, there are at least two Nash equilibria and thus the local monotonicity 
condition is necessary. 

G. Proof of Lemma 

Suppose ^ E V{C) such that there exists Li < L2 < with /i(-Li) > and fi{Ls) > 0. Choose 
< a, l3 < such that aLi + /3L3 = (a + (3)L2. Then, we construct a new mixed strategy ft, as 

= - a, fliL^) = /i(L3) - (3, fl{L2) = /^(La) + a + (3, 

and jl(L) = fi{L) for L ^ Li, L2, L^. By the definition of and convexity, 

u^{x) - Uji{x) = ax^^ + /3x^^ - (a + /3)x^^ > 0. 

Therefore, r^(k) < r^{k), A; > 2. It follows then 

Csum(/i) < A ^ +CsL \ < Csum(/i), 

L=l \A:=0 / 

where the second inequality follows from the strict convexity of E[Vr(L, /x)] in L. Therefore, /i 7^ n*^^. 

H. Proof of Lemma |9| 

First, we show the continuity of E[iy(Lj(c), L_i)] in L^i{-). Suppose L^""] L^i as n — )■ 00. By 
the definition of metric, we have fi (n) — )■ /iL_r Since ¥.[W{Li{c), L_i)] = E[W{Li{c), fii^-)], by the 

— i 

continuity of E[iy(Lj(c), with respect to /iL_i, we conclude that E,[W{Li{c), L_i)] is continuous 

in 

Next, we show that the correspondence BR(L_j) is in fact a function. Let Li E BR(L_j) and 
= Cq < Ci < ■ ■ ■ < cl,-,-,^^ = Cmax dcnotc the jumping points of Lj. It follows from Fig. [6] that for 
j = 1, . . . , Ljnax ^ 1' Cj is uniquely determined by 

_ Cs 

~ E[Wij,L_,)]-E[Wij + l,L_,)y 
Therefore, Li is unique and BR(L_i) is a function from S to 5. 
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Finally, we show the continuity of BR(L_,;). Suppose L^^} — )• L_j, L-"^ = BR(L^^''), and Li = 
BR(L_j), we prove that L^'^^ Li. Denote the jumping points of L,-"^ and Lj by = Cq"^ < c[^^ < 
■ ■ ■ < c^"2ax ^ '^max and = Co < Ci < ■ ■ ■ < cl^^^ = Cinax respectively. It suffices to show that 
cj"-* — )■ Cj, for j = 0, . . . , Ljnax- From Fig. [61 we can see that for j = 1, . . . , Lmax — 1, 

"E[iy(j,L5)]-E[l^(j + l,L5)]' 
By the continuity of E[iy(L,L_j)] with respect to L_i{-), 

cf^ Cj, for 1 < J < L max — 1, 

which further implies that /i^(n) — )■ /i/,. and concludes the proof. 

/. Proof of Theorem |6| 

Define the jumping measure as 

L=l 

It follows that II J(^, a;)||Tv < ALmax + 1- Also, 

II J(e,a;) - Jiv,x)\\TV = A | ■), ^^^"^ - r^®^-^) |< AL^axlle^^"' - r^^^-^TV, 

L=l 

_ ^^L^ly^ < AL^ax(i:max " 1)11^ " ^IItV- 

Therefore, ||J(^,x) — J(?7,a;)||Tv < ALmax(-^max — 1)||'C ~ '7||tv, which concludes the proof using 
Proposition 2.3 in 

J. Proof of Theorem 

It is proved using a method developed by Sznitman [fTSll with three essential steps: prove that 

1) i^iu^))N>i is tight in P(P(D(M+,N))). 

2) the non-linear martingale problem (fTTI) is satisfied by any probability measure in the support of 
any accumulation point of {Jf{u^))]\f>i. 

3) the non-linear martingale problem (fTTj) has a unique solution T starting at any 7 in V(N). 
Once these three steps are done. Step 2 and Step 3 imply that the only accumulation point of 
(=Sf(z/^))jv>i is Sr, and Step 1 implies that (=Sf (z/^))jv>i weakly converges to 5r- Then, by Proposition 
2.2 in [fTSl . the conclusions follows. The individual steps are proved as follows. 
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Step 1. It is equivalent to prove that (^(Qf ))Ar>i is tight in P(D(M+, N)), see Proposition 2.2 in 
[fTSl . The jumps of are included in those of a Poisson process A'^ of rate ALmax + 1, and their 
size is bounded by 1, so the modulus of continuity in D(R_|_,N) of is less than that of . The 
law of A^ does not depend on > 1, and since {Qq)n>i converges and hence is tight, the basic 
tightness criterion in Ethier and Kurtz IfTTl holds. 

Step 2. Let be the law of and n°° G P(P(D(M+, N))) be an accumulation point. Take 
0<si<---<Sk<s<t and a bounded function g on N^, and then 

^ L=l 



+ 1q„>i0 iQu)}dujg{Qs,,...,QsJ,R) 
is n°°-a.s. continuous. Set g'^'^ = g{Qsi, • • • , QsJ and Yg^t = Yt — Ys for a process Y. It follows that 



1e 

AT 



E 



<t>,i,N\i,N\2 



)9' 



N 



N 
N -1, 



')9 



-E 



4>,l,N (f),l,N 



'S,t 



and then Lemma [H] implies that (G^ H^) = 0(1/A^). The Fatou lemma implies that (G^n°°) < 
limjv^oo(G'^, n^) = and thus G{R) = 0, n°°-a.s. Since this holds for arbitrary bounded g and 
< si < ■ ■ ■ < Sfc < s < t, we conclude that R solves the nonlinear martingale problem (fTTl) . n°°-a.s. 
Step 3. This result is proved in Theorem [6l 



K. Proof of Lemma [73] 

First, note that increasing rQ^k) only increases all rt{k), because drt{k)/dt is non-decreasing in 
rt{j) for j ^ k [fTSll . Therefore, it suffices to show the conclusion for the following two cases: 
ro{k) > r{k),\/k and ro{k) < r{k),yk. 

Second, define vf{k) = J2f>k ^tO) ^^'^ 'Vt{k) = V^ik). We show that for the above two cases, Vt{k) 
is uniformly bounded over all t and k. If ro(A;) < r{k),yk, since r(k) is a fixed point, rt{k) > r{k) 
for all t. Therefore, 

i>i i>i 
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If ro{k) > r{k),yk, then rt{k) > r{k) for all t. From the mean field equation (fT3]) . 

^max 



dt 



A - A ^ mrt{K) - (r,(l) - n{K + 1)) < A. 



L=l 



It follows that ff'(l) < At and thus < At, which further implies that YrnvK^oo^tiK) = 0. 

Therefore, taking limit K oo over both sides of the above equation gives 

dvt{l) 



dt 



A-n(l) =r(l) -rt(l) < 0. 



Thus, vtik) < vt{l) < vo{l). 

Finally, in order to show that limt^oo i"tik) = r{k), it suffices to show 



-oo < 



{rt{k) - r{k)) < oo, 



because under the above two cases, the sign of {rt{k)—r{k)) does not change in t. We prove the second 
inequality by induction and the first inequality can be proved in the same way. The first inequality 
trivially holds for /c = since rt(0) = r(0) = 1. Now suppose it holds for k — 1. From mean field 
equation (fT3] ) and definition of r{k), 

-^max -^max 



dvtjk) 
dt 



A KL)r^ik - 1) - n{k) = A J2 {r^{k - 1) - r\k - 1)) - {n{k) - r(fc)). 



L=l 



L=l 



By integrating it, it follows that 



vt{k) = vo{k) + 



A £ ^^{L) {r^{k - 1) - r\k - 1)) - (r,(fc) - r{k)) 



L=l 



ds. 



By induction, J^{rt{k — 1) — r{k, — \))dt < oo. Therefore, 



L=l 



oo 



ds < Lmax / {rt{k - 1) - r{k - l))dt < oo. 





By assumption, Vo{k) is bounded and we just proved that Vt{k) is uniformly bounded in t. Thus, 
j^{rt{k) — r{k))dt < oo and the conclusion follows. 



L. Proof of Theorem \ 

Let G P(D(M+, N)) denote the stationary measure on the path space of the finite model. 

Step 1. Prove that (^st(Q^))Af>i is tight. It is equivalent to prove that {^st{Qi{^)))N>i is tight. 
Let system be the finite model with all the customers sampling one queue; while system 1 be 
the finite model with all the customers using strategy /i. The system is an i.i.d. system. If both 
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A 



systems are started at the same initial value, with law being the stationary distribution of system 0. 
Then, by the coupling result in Theorem [8l 

N N 
i=l i=l 

Since A < 1, by the ergodicity result in Theorem |9] and Fatou lemma, 

^stlQ^m < lim inf E[Q^'\t)] < ^ 



which implies that for all N>1, ^st{Qi{0)){[K, oo)) < A(l - Xy^R-^ and hence (0))jv>i 
is tight. 

Step 2. Prove that JfgtiQo) weakly converges to Let Iljf be an accumulation point of JfstiQo)- 
Suppose the finite model starts with the stationary distribution. Let denote the law of and n°° 
denote an accumulation point of 11^ . Then, we can prove exactly as for Theorem |7] that 11^ is tight, and 
that any law R in the support of U°° satisfies the non-linear martingale problem. In particular, {Rt)t>o 
solves the nonlinear Kolmogorov equation. Since the initial distribution is the stationary distribution, 
= Uf. Taking the limit along the subsequence, it follows that = 1^°. 

Take e > and an open neighborhood F of 7^. For j G N, let Vj be the set of all r] in V(N) such 
that the solution for the Kolmogorov equation ([T2)) starting at r] is inV for all times t > j. By Lemma 
[T3l P(N) = UjVj and since Vj C Vj+i, there is k such that U^{Vk) > 1 - e. It follows that 

U^{V) = U^{V) = U^{Rk eV)> U^{Ro G Vk) = U^iVk) > 1 - e. 

Therefore, 11^ = 

Step 3. Since converges in law to 7^, by Theorem |71 the conclusions follow. 
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